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ARI  Research  Reports  and  Technical  Reports  are  intended  for  sponsors  of 
R&D  tasks  and  for  other  research  and  military  agencies.  Any  findings  ready 
for  implementation  at  the  time  of  publication  are  presented  in  the  last  part 
of  the  Brief.  Upon  completion  of  a  major  phase  of  the  task,  formal  recom¬ 
mendations  for  offidial  action  normally  are  conveyed  to  appropriate  military 
agencies  by  briefing  or  Disposition  Form. 


FOREWORD 


The  Presidio  of  Monterey  Field  Unit  has  conducted  a  broad  program  of 
research  on  problems  of  training  in  Army  field  units  since  its  establish¬ 
ment  in  1974.  This  program  has  resulted  in  a  number  of  cost  effective 
solutions  to~Afmy~1nraining-  problems  in  a  number  of  different  areas.  The 
Unit  Training  Programs  Team  concentrates  on  issues  relating  to  efficient 
use  of  training  resources,  integration  of  individual  and  collective  training 
and  the  development  and  execution  of  cost  effective  training  programs  by 
units . 

Current  training  doctrine  in  units  as  embodied  in  the  Battalion  Training 
Management  System,  directs  trainers  to  conduct  proficiency  testing  before 
individual  or  collective  training  is  executed  (i.e.,  pretesting).  However, 
pretesting  is  rarely  conducted  in  the  field,  and  trainers  typically  challenge 
the  cost  effectiveness  of  such  a  procedure  as  a  prerequisite  for  individual 
skill  training. 

This  report  describes  a  methodology  that  was  developed  for  determining 
the  cost-effectiveness  of  any  form  of  pretesting  procedure.  The  methodology 
was  tested  through  its  application  to  empirical  data  and  found  to  be  workable. 
The  report  describes  how  this  methodology  can  be  employed  to  improve  the 
efficient  use  of  time  and  resources  for  unit  training  through  pretesting. 

The  methodology  can  be  easily  used  by  training  managers  in  FORSCOM,  USAREUR, 
and  other  organizations  responsible  for  conducting  training. 
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A  METHODOLOGY  FOR  ESTIMATING  RELATIVE  COST-BENEFITS  OF  ALTERNATIVE 
PRETESTING  PROCEDURES 


EXECUTIVE  SUMMARY 


Requirement : 


To  develop  a  quantitatively  based  methodology  which  will  enable  objec¬ 
tive  selection  of  an  optimally  effective  pretesting  procedure  from  a  set  of 
alternative  procedures. 


Procedure : 


The  research  was  accomplished  as  follows :  Alternative  pretesting  pro¬ 
cedures  were  formulated.  Variables  that  affect  the  amount  of  time  saved  or 
lost  by  employing  pretests  were  identified  and  defined.  Algebraic  cost/ 
benefit  models  which  take  into  account  measurement  accuracy,  pretesting  time, 
and  training  time  were  constructed  sq  that  the  amount  of  time  saved  (or  lost) 
by  pretesting  could  be  estimated.  A  limited  sample  of  empirical  data  was 
gathered  and  analyzed  to  illustrate  the  possible  applications  of  the  cost/ 
benefit  models. 


Findings : 

It  was  found  possible  to  construct  generalized  algebraic  models  that  can 
be  used  to  calculate  the  time  saved  or  lost  (i.e.,  a  benefit  value)  when  any 
particular  pretesting  procedure  is  employed  in  a  group  training  situation. 

By  simple  comparison  of  the  benefit  values  for  each  of  the  alternative  pro¬ 
cedures,  an  optimal  procedure  can  be  identified.  Analysis  of  a  small  sample 
of  data  for  infantry  tasks  implied  that  considerable  improvements  in  time 
utilization  may  be  achieved  through  use  of  carefully  designed  pretesting 
procedures . 


Utilization  of  Findings: 

This  cost/benefit  methodology  will  be  used  to  identify  an  optimal  pre¬ 
testing  procedure  for  the  common  tasks  required  by  MOS  11B  at  Skill  Level  1. 
Based  on  the  logic  of  the  cost/benefit  analysis,  and  the  preliminary  empiri¬ 
cal  results,  the  use  of  pretesting,  where  appropriate,  is  being  advocated  by 
the  Army  Training  Board. 
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A  METHODOLOGY  FOR  ESTIMATING  RELATIVE  COST-BENEFITS  OF 
ALTERNATIVE  PRETESTING  PROCEDURES 


INTRODUCTION 

The  costs  of  education  and  training  are  one  of  the  largest  and  most  visi¬ 
ble  aspects  of  both  civilian  and  military  budgets.  In  the  press  to  economize, 
the  U.S,  Army  has  been  searching  for  methods  which  may  achieve  increased 
training  efficiency  as  well  as  higher  standards  of  effectiveness.  One  re¬ 
cently  adopted  feature  of  Army  training  doctrine  aims  at  both  efficiency  and 
effectiveness.  Specifically ,  this  feature  is  the  requirement  to  conduct  per¬ 
formance-based  diagnostic  testing  before  training  on  a  task  is  given,  i.e., 
pretesting . 

In  theory,  pretesting  should  yield  the  following  advantages: 

a.  Training  time  will  be  effectively  gained  by  eliminating  unnecessary 
training  sessions. 

b.  Planning  and  scheduling  of  training  will  be  facilitated  when  pre¬ 
tests  are  used  to  identify  common  deficiencies; 

c.  Testing  (and  training)  standards  will  be  maintained  at  high  levels 
when  the  pretests  are  performance-based  (i.e.,  actually  performing 
a  task  instead  of  talking  or  writing  about  it) ; 

d.  Infrequently  used  skills  will  be  reinforced  by  the  practice  gotten 
from  talking  the  performance-based  pretests; 

e.  Job  climate  will  be  improved  by  avoiding  unnecessary,  boring  training. 

The  purpose  of  the  research  reported  here  was  to  develop  a  methodology  for 
estimating  the  time  savings  produced  by  various  forms  of  pretesting  so  that  the 
most  cost-effective  form  could  be  identified. 

Pretesting  has  been  incorporated  into  an  on-the-job  training  system  for 
infantry  currently  undergoing  research  and  development  by  the  Army  Research 
Institute  (Bialek  et  al.,  1978).  In  this  experimental  system,  the  rifle  squad 
leader  is  designated  as  the  key  trainer,  and  it  is  his  responsibility  to  con¬ 
duct  the  pretests.  However,  the  amount  of  time  that  may  actually  be  saved  by 
having  squad  leaders  conduct  pretests  is  uncertain. 

Consider  the  following  hypothetical  situation.  A  squad  leader  with  a 
nine-man  group  intends  to  have  all  of  his  men  become  proficient  in  performing 
some  individual  task.  It  happens  that  he  has  insufficient  information  con¬ 
cerning  any  of  his  men's  ability  to  perform  the  task,  so  he  conducts  a  pre¬ 
test.  The  pretest  chosen  is  a  performance  test  that  may  be  administered  to 
only  one  man  at  a  time  and  it  requires  about  10  minutes  to  conduct  for  each 
man.  Altogether  then,  conducting  this  pretest  for  the  entire  squad  consumes 
l1}  hours  of  the  trainer's  time,  and  during  this  period  the  squad  members  may 
not  be  spending  their  time  usefully.  If  it  turned  out  that  every  squad  mem¬ 
ber  failed  the  pretest  and  needed  substantial  training,  then  no  time  was  saved, 
and  1*5  hours  were  lost. 
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Given  that  pretesting  may  not  necessarily  yield  time  savings  and  that 
different  forms  of  pretesting  may  also  differ  in  their  cost-effectiveness,  a 
need  becomes  evident  for  research  to  develop  a  methodology  for  measuring  and 
predicting  the  cost-effectiveness  of  alternative  pretesting  procedures  (in¬ 
cluding  the  procedure  which  deletes  pretesting  of  any  kind,  i.e.,  the  null 
pretest)  so  that  an  optimal  procedure  may  be  selected  for  a  given  situation. 
The  research  that  was  performed  to  meet  this  need  will  be  described  according 
to  the  following  stages: 

a.  Alternative  pretesting  procedures  were  designed. 

b.  Variables  which  may  affect  the  time  saved  or  lost  by  employing  the 
pretesting  procedures  (including  the  null  pretest)  were  identified. 

c.  Cost-benefit  models  were  formulated  in  terms  of  training  time  saved 
or  lost. 

d.  An  efficient  data  collection  procedure  was  devised. 

e.  The  cost-benefit  models  were  applied  to  sample  data. 

Each  of  these  stages  is  described  below. 


DESIGN  OF  ALTERNATIVE  PRETESTING  PROCEDURES 

The  following  procedures  were  considered  for  possible  use  as  pretests: 

a.  Each  squad  member  estimates  his  own  task  proficiency; 

b.  The  squad  leader  estimates  the  task  proficiency  of  his  squad's 
members ; 

c.  Paper-pencil  criterion-referenced  tests  are  used; 

d.  Computer  or  equipment  intensive  simulation  is  used  for  testing; 

e.  Hands-on  performance  testing  is  used. 

Self-estimates ,  although  of  uncertain  validity,  are  fast  and  easy  to  ac¬ 
quire.  Given  the  high  levels  of  turbulence  found  in  operational  units,  the 
infrequent  occurrence  of  many  infantry  tasks,  and  the  fallibility  of  memory, 
estimation  by  the  trainer  (i.e.,  squad  leader)  was  rejected  for  this  research 
effort.  Paper-pencil  tests  are  relatively  quick  and  easy  to  administer,  but 
have  uncertain  validity  for  infantry  troops  due  to  their  verbal  requirements. 
Simulated  testing  was  rejected  due  to  resource  requirements  beyond  the  scope 
of  this  project.  Hands-on  performance  testing  is  typically  time  consuming, 
especially  for  process  testing  which  requires  observation  of  each  task  step. 
However,  the  results  of  performance  testing  provide  the  criterion  measure  of 
task  proficiency  for  all  practical  purposes.  In  summary,  a  decision  was  made 
to  explore  the  use  of  self-estimation,  paper-and-pencil  criterion-referenced 
tests,  and  performance  tests. 
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One  possible  approach  to  using  the  three  candidate  pretests  would  be  to 
employ  each  one  by  itself.  For  example,  a  soldier  could  be  asked  if  he  is 
able  to  perform  a  task  to  standard,  and  then  either  be  placed  in  training  if 
he  said  "no,"  or  assigned  to  some  other  activity  if  he  said  "yes."  Another 
possible  approach  is  to  arrange  the  candidate  pretests  in  a  systematic  order 
to  capitalize  on  their  virtues  while  minimizing  the  effects  of  their  weak¬ 
nesses.  Figure  1  shows  an  ordering  of  the  pretests  which  may  provide  an  op¬ 
timal  procedure.  This  procedure  provides  the  easiest  and  fastest  pretest  as 
the  first  step,  the  second  easiest  to  administer  as  the  second  step,  and  the 
most  accurate,  but  time  consuming,  pretest  (the  hands-on  performance  test) 
as  the  last  step  for  anyone  who  is  not  already  eliminated. 

Requiring  soldiers  to  take  the  performance  test  as  the  last  step  insures 
that  no  one  will  falsely  be  considered  proficient,  since  the  soldier  must 
demonstrate  his  proficiency  before  being  cr'.dited  with  a  "Go."  Theoretically 
the  only  error  that  can  be  made  with  the  aoove  ordering  of  pretests  is  to  as¬ 
sign  a  soldier  to  training  when  he  does  not  need  it.  Such  errors  may  occur 
either  because  the  soldier  misjudges  his  true  ability,  or  because  he  fails 
the  paper-and-pencil  test. 

In  addition  to  the  pretesting  procedure  shown  in  Figure  1,  several  other 
possible  procedures  were  defined  for  this  research.  These  were: 

a.  Self-estimate  followed  by  performance  test  (see  Figure  2) ; 

b.  Paper-and-pencil  test  followed  by  performance  test  (see  Figure  3) ; 

c.  Performance  test  alone  (see  Figure  4) ; 

d.  No  pretei ting,  everyone  enters  training. 


COST-BENEFIT  ANALYSIS  MODEL  FOR  SQUAD  MEMBER  TIME 

Having  defined  alternative  pretesting  procedures,  it  is  necessary  to  de¬ 
vise  a  method  for  identifying  an  optimal  pretesting  procedure  for  various 
training  situations.  The  approach  taken  was  to  identify  variables  which  may 
influence  time  saved  or  lost  when  squad  members  are  pretested,  and  then  to 
construct  an  algebraic  equation  which  may  be  used  to  calculate  the  time  saved 
or  lost  by  using  the  diagnostic  information  produced  by  testing.  In  addition 
to  the  time  saved  or  lost  for  the  squad's  members,  the  use  of  pretesting  may 
also  affect  the  time  available  to  the  squad's  leader.  However,  since  squad 
leader  time  has  a  different  value  than  squad  member  time,  and  since  there  are 
approximately  10  times  more  squad  members,  the  cost-benefits  of  pretesting 
for  each  kind  of  soldier  will  be  dealt  with  separately.  In  the  following 
portions  of  this  paper,  the  cost-benefit  analysis  for  squad  member  time  will 
be  developed,  after  which  the  analyses  for  the  squad  leader  will  be  presented 

It  should  be  stressed  that  the  analysis  presented  below  reflects  the  re¬ 
quirement  that  a  soldier  must  be  performance  tested  before  he  can  be  classi¬ 
fied  "GO."  As  a  consequence,  the  only  testing  error  is  to  classify  a  GO  sol¬ 
dier  NO  GO  during  a  pretest.  Assuming  the  performance  test  is  accurately 
scored,  a  NO  GO  soldier  will  not  be  misclassified  GO. 
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FIGURE  4 

PRETESTING  PROCEDURE  D 


I 

TRAIN  1 
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Tiu-  training  model  underlying  the  analysis  is  as  follows: 

1.  A  pretest  of  some  sort  is  qiven  (although  for  certain  tasks,  safe'  y 
considerations  rule  out  pretesting) . 

.  for  most  tasks  to  be  taught ,  a  complete,  un  inter  rapt  ed  demon. -t :  a*  n 
of  tiie  task  is  qiven;  then  each  task  step  is  in  turn  shown  a:,  i  > 
plained  —  step  by  step.  Where  practical,  the  'raisers  f  1  1  ,v.  ..iJos  i, 
performinq  each  step. 

The  trainees  then  attempt  to  perform  the  task,  with  .-each  i.vi  a'.u »  i  - 
iblo;  trainers  are  to  stop  the  trainee  whenevoi  he  makes  a  mistake  , 
and  show/explain  so  that  errors  are  not  trainee,  but  only  oori'  .  • 
skill . 

■  I.  Whenever  the  trainee  or  trainer  believes  the  trainee  lias  "learned " 
l.o  perform  the  task  to  standard,  a  performance  test  for  the  record 
is  given  (training  records  are  kept  to  aid  training  management  de¬ 
cisions).  If  the  trainee  fails,  his  errors  are  corrected,  and  de¬ 
pending  on  the  level  of  performance  he  is  either  retested,  or  looped 
back  to  step  i  to  continue  practicing. 

fret os ting  Vari able.: 

The  key  variables  that  have  been  employed  to  perform  a  cost-benefit  anal 
<f  pretesting  are  defined  below: 

B  =•  Benefit,  defined  as  the  total  number  of  man-minutes  saved 

by  applying  any  pretesting  procedure. 

q  =  The  proportion  of  squad  members  (excluding  tin  squad  leader 

who  serves  as  the  trainer/tester)  who  are  able  to  perform 
the  task  ( i . o .  ,  who  are  "GO")  before  training  is  given. 

e  -  The  proportion  of  men  in  a  squad  who  are  able  to  perform 

the  task  but  are  erroneous ly  classified  by  a  pretesting 
procedure  as  "NO  GO." 

h  =  q  -  e  =  The  proportion  of  men  in  a  squad  who  are  correctly  classi¬ 
fied  by  a  pretesting  procedure  as  "GO”  on  a  task  (hits) . 

P  =  The  time  that,  it  takes  for  an  entire  squad  to  be  protested. 

<'  ~  The  time  it  takes  for  one  soldier  to  undeiqo  a  performance 

test  which  is  termed  a  "checkout  ." 

1)  =  Time  for  the  demonstration/explanation  phase  of  training. 

The  demonstration  is  a  complete  uninterrupted  task  perfor¬ 
mance.  The  explanation  consists  of  a  step-by-step  demon¬ 
stration  with  each  step  explained. 

N  -  The  number  of  men  in  a  given  squad  (excluding  the  squid 

leader  who  serves  as  the  t  i  .i  inet /t  ester )  . 
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Primary  Time  Saved  by  Pretesting.  The  principle  segment  of  time  that 
may  be  saved  by  conducting  a  pretest  is  the  time  that  it  takes  to  demonstrate 
and  explain  how  a  given  task  is  performed  (D) .  Soldiers  who  are  found  by  a 
pretest  to  be  proficient  in  a  task  do  not  have  to  watch  the  demonstration/ 
explanation,  but  may  instead  engage  in  a  more  productive  activity,  such  as 
serving  the  trainer  as  a  peer  instructor  or  demonstrator ,  or  working  on  an¬ 
other  task,  or  taking  care  of  a  personal  need. 

For  the  purpose  of  constructing  a  model  describing  how  time  is  saved  or 
lost  for  squad  members,  it  is  assumed  here  that  any  soldier  may  take  a  per¬ 
formance  test  immediately  after  the  task  demonstration  is  finished.  (This 
assumption  is  clearly  consistent  with  the  envisioned  training  system  but  rep¬ 
resents  a  worst  case  situation  for  estimating  the  positive  value  of  pretest¬ 
ing,  since  in  reality,  soldiers  may  have  to  wait  to  be  tested  while  the  squad 
leader  coaches  soldiers  who  need  help.)  In  this  case,  the  length  of  the 
demonstration  (D)  determines  the  amount  of  time  that  can  be  saved  by  an  ac¬ 
curate  pretest.  The  time  that  the  pretest  costs  will  be  dealt  with  later. 

Assume  for  the  sake  of  argument  that  two  soldiers  who  could  both  perform 
a  task  had  to  sit  through  a  demonstration  because  a  pretest  was  not  given,  or 
it  was  given  but  falsely  found  them  to  be  "NO  Go's."  Each  soldier  would  have 
lost  D  time.  In  addition  to  the  D  time  lost  for  each  soldier,  it  may  be  seen 
that  the  second  soldier  to  be  tested  after  the  demonstration  has  finished  has 
also  lost  the  time  that  it  took  for  the  first  soldier  to  be  performance  tested 
immediately  after  the  demonstration.  This  time  loss  that  is  incurred  as  one 
or  more  soldiers  wait  for  their  turn  to  be  tested  occurs  in  the  same  way  for 
both  pretesting  and  posttesting.  We  have  accordingly  adjusted  the  time  cost 
of  pretesting  when  constructing  the  cost-benefit  model,  and  will  explain  how 
after  the  pretest  cost  is  defined  below. 

Algebraically,  the  total  amount  of  time  in  man-minutes  that  is  saved  by 
having  proficient  soldiers  avoid  a  demonstration  that  they  do  not  need  is 
found  by  multiplying  the  demonstration  time,  D,  times  the  total  number  of  men 
in  a  given  squad  who  have  been  correctly  classified  as  "GO"  by  the  pretest, 

Nh,  or  finally: 

Primary  Savings  =  NhD 

Pretesting  Cost.  The  direct  time  cost  for  pretesting  is  simply  the 
span  of  time  that  it  takes  the  squad  leader  or  trainer  to  pretest  his  en¬ 
tire  squad,  times  the  number  of  men  in  the  squad,  or: 

Uncorrected  Pretest  Cost  =  -NP 


Adjustment  of  Pretest  Cost.  In  the  ne*  training  system  that  underlies 
the  analysis  being  developed  here,  a  performance  test  must  be  administered  to 
each  soldier  to  establish  if  he  is  "GO"  or  not  when  the  soldier  or  his  trainer 
believes  he  has  mastered  the  task.  In  other  words,  a  performance  test,  called 
a  checkout,  must  be  administered  to  each  soldier  as  part  of  the  training  pro¬ 
cedure  in  the  new  system.  Thus,  the  cost  of  the  performance  portion  of  the 
pretest  procedure  for  a  soldier  who  passes  it  is  not  uniquely  attributable  to 
the  pretest  procedure.  The  reason  is  this  cost  would  have  been  incurred  any¬ 
way  when  the  performance  test  was  given  after  the  task  demonstration.  There¬ 
fore,  the  time  cost  for  successful  performance  of  the  checkout  during  the 
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pretest  procedure  is  deducted  from  the  overall  cost  of  the  pretest  procedure. 
The  analysis  of  this  time  credit  is  explained  below. 

The  number  of  men  in  a  squad  who  are  correctly  measured  to  be  "GO"  on  a 
task  is  defined  above  as  Nh. 

The  time  cost  in  man-minutes  when  one  member  of  this  group  is  adminis¬ 
tered  the  checkout,  C,  is  the  time  C  multiplied  by  the  group  size,  Nh,  since 
everyone  in  this  group  is  held  up  while  the  trainer  administers  the  checkout 
to  one  man.  This  time  cost  is: 


CNh 

Since  each  member  of  this  "GO"  group  must  take  the  performance  test,  the 
total  time  involved  in  man-minutes  is  found  by  multiplying  the  group  size  by 
the  cost  for  administering  one  checkout: 

2  2 

ChCNh ,  or  CN  h  . 

Thus,  the  cost  of  pretesting,  adjusted  for  the  posttraining  checkouts  that 
would  have  been  given  even  if  pretesting  were  not  accomplished  is: 

2  2  2 

Pretest  Cost  *  - (NP  -  CN  h  ) ,  or  -N9P  -  NCh  ) . 

Time  Lost  When  a  Pretest  Fails  To  Identify  a  "GO"  Soldier.  When  a  pro¬ 
ficient  soldier  is  misclassif ied  as  "NO  GO"  by  a  pretest,  for  whatever  rea¬ 
son,  he  will  then  be  required  to  attend  a  task  demonstration/explanation  ses¬ 
sion  that  he  does  not  need.  By  spending  his  time  on  the  unnecessary 
demonstration,  other  tasks  which  could  have  been  attended  to  go  undone,  and 
thus  time  in  this  sense  is  wasted. 

It  might  be  argued  that  the  time  spent  on  the  demonstration  is  not  en¬ 
tirely  wasted,  since  the  proficient  soldier's  ability  to  perform  well  and  to 
retain  his  skill  are  incremented  to  some  degree.  However,  there  are  several 
reasons  for  believing  that  any  such  increment  is  generally  negligible.  These 
reasons  derive  from  considerations  of  motivation  and  cognitive  theory.  A 
soldier  who  perceives  that  he  already  knows  how  to  perform  a  task  that  is 
being  demonstrated  is  not  likely  to  watch  closely,  if  at  all.  Rather,  he  may 
experience  dissatisfaction  and  boredom,  with  consequent  inattention  and  day¬ 
dreaming.  From  a  learning  theory  point  of  view,  the  relatively  passive 
demonstration/explanation  session  has  a  serious  limitation — the  cognitive 
requirements  for  watching  someone  else  do  something  are  different  from  those 
for  actively,  smoothly  performing  a  task,  so  that  transfer  of  learning  from 
watching  to  doing  is  problematical;  furthermore,  this  problem  will  be  com¬ 
pounded  by  inattention. 

The  proportion  of  squad  members  who  are  mistakenly  classified  as  "NO  GO” 
is  -n  so  that  the  total  number  of  soldiers  involved  is  Nen-  The  time  wasted 
by  providing  these  soldiers  with  the  demonstration  they  do  not  need  is  simply 

Wasted  Time  *  -Ne  D. 

N 

Complete  Cost-Benefit  Model.  Collecting  the  terms  established  above 

for: 
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a.  The  primary  time  savings  from  pretesting; 

b.  The  cost  of  pretesting  adjusted  for  the  checkouts  that  would  be 
given  anyway;  and 

c.  The  time  wasted  when  a  pretest  fails  to  identify  proficient  soldiers, 
we  have : 


B  =  NhD  -  N{P  -  NCh2)  -  Ne^D 
By  factoring  out  "N,"  and  rearranging  terms  we  get: 

General  Cost-Benefit  Model:  B  =  N(D(h  -  e^)  -  P  +  NCh2) 


SPECIFIC  COST-BENEFIT  MODELS 

Having  constructed  a  generalized  benefit  model,  the  next  step  is  to  de¬ 
rive  a  specific  form  of  the  equation  for  each  of  the  specified  alternative 
pretesting  procedures. 


Equation  for  Procedure  A  (Figure  1) 

Procedure  A  involves  all  three  pretest  elements — self-estimate,  written 
test,  and  checkout.  It  is  necessary  to  define  the  following  new  variables  to 
account  for  these  elements. 

y  *  The  proportion  of  squad  members  who  say,  "Yes ,  I  can  perform  the 
task  to  standard." 

A  =  The  time  it  takes  a  squad  to  be  asked  for  their  self-estimates  plus 

the  time  to  give  the  squad  leader  the  answers. 

w  *  The  proportion  of  squad  members  who  pass  the  written  test. 

K  =  The  time  it  takes  to  administer  and  score  the  written  knowledge 

test  for  a  squad. 

Two  forms  of  the  benefit  equation  need  to  be  defined  to  reflect  two  pos¬ 
sible  outcomes  for  the  self-estimates. 

a.  Equation  for  Procedure  A  Where  y  =  0.  The  time  cost,  P,  involves 
only  the  time  it  takes  to  obtain  the  self-estimate.  Since  the  checkout  will 
not  be  given  to  any  squad  member,  no  possible  "GO"  members  can  be  identified. 
Therefore  h  =  0. 

Substituting  A  for  P,  and  0  for  h  yields: 

For  y  =  0,  B  *  N(D(0  -  eN)  -  A  +  NCO)  =  N(-DeN  -  A) 
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Inspection  of  this  equation  reveals  the  following  points: 


1.  The  benefit  value  cannot  be  positive  since  both  terms  are  negative. 

2.  The  least  costly  outcome  is  the  case  where  the  error  term  is  zero 
(i.e.,  all  soldiers  correctly  estimated  that  they  couldn't  perform 
the  task)  in  which  case  the  only  cost  stems  from  A. 

b.  Equation  for  Procedure  A  Where  y  >  0.  The  time  required  for  pre¬ 
testing,  P,  has  three  components,  one  for  each  of  the  three  pretest  elements: 

P ' =  A  +  K  +  NwC 

The  term  K  mainly  reflects  the  time  that  it  takes  for  the  slowest  squad 
member  to  take  the  written  test,  where  scoring  is  quickly  done  with  a  key  or 
from  memory.  The  term  NwC  reflects  the  number  of  squad  members  who  have 
passed  the  written  test,  Nw,  multiplied  by  the  time  required  to  give  each 
one  the  checkout,  C. 

Substitution  for  P  in  the  general  model  yields: 

For  y  >  0,  B  =  N(d(h  -  eN)  -  (A  +  K  +  NwC)  +  NCh2) . 


Equation  for  Procedure  B  (Figure  2) 

The  term  P  has  two  components — one  for  the  self -estimate.  A,  and  one  for 
the  time  that  it  takes  to  give  the  checkout  to  each  soldier  who  estimated  he 
could  perform  the  task,  NyC.  Substitution  for  P  yields: 

B  =  N(D(h  -  eN)  -  (A  +  NyC)  +  NCh2) . 

It  may  be  observed  here  that  acquisition  of  positive  values  for  h  is  crucial 
for  generating  positive  values  for  B.  If  h  is  zero  (which  may  occur  when  no 
one  can  perform  a  task  before  training,  or  none  of  the  soldiers  who  can  per¬ 
form  the  task  estimate  they  can  do  it)  then  NCh2  drops  out,  and  the  remaining 
terms  cannot  rise  above  zero.  In  addition,  the  largest  possible  benefits 
arise  when  the  demonstration  time  is  large,  as  well  as  h  equaling  one. 

Equation  for  Procedure  C  (Figure  3) 

The  term  P  has  two  components,  one  for  K  and  one  for  C.  Substituting 
for  P  yields: 

B  =  N(D (h  -  e.J  -  (K  +  NwC)  +  NCh2). 

N 

Equation  for  Procedure  D  (Figure  4) 

The  value  for  P  is  derived  solely  from  the  size  of  the  squad  and  the 
time  for  conducting  each  squad  member's  checkout: 


B  =  N  (D  (h  -  e  )  -  NC  +  NCh2)  ,  or 
N 

N(D(h  -  e  )  -  NC (1  -  h2)) . 

N 

Equation  for  No  Pretest 

When  no  pretest  is  given,  P  is  zero.  Furthermore,  there  is  no  possi¬ 
bility  of  identifying  any  proficient  soldier  before  the  demonstration  is 
given,  so  the  proportion  of  hits,  h,  is  zero. 

The  resultant  equation  is: 


An  equivalent  alternative  equation  is  based  on  the  fact  that  the  magni¬ 
tude  of  the  error  term  eN  equals  the  porportion  of  squad  members  who  are 
"GO,"  g,  before  the  demonstration  is  given.  The  alternative  equation  is: 

B  =  -NDg. 

Therefore,  when  no  pretest  is  given  no  time  can  be  saved,  and  time  is 
lost  as  a  direct  function  of  the  length  of  the  demonstration/explanation  and 
the  number  of  soldiers  who  are  proficient  without  further  training. 


Modification  to  the  Equations  for  Simultaneous  Checkouts 

Performance  testing  for  the  majority  of  tasks  covered  by  the  new  training 
system  requires  observation  of  the  step-by-step  process  used  by  each  soldier, 
since  there  is  no  clear-cut  product.  In  some  cases,  even  where  evaluation  of 
a  product  would  suffice  for  a  checkout,  the  value  of  the  diagnostic  informa¬ 
tion  that  may  be  gotten  from  observation  of  performance  is  so  high  that  a 
process  test  is  preferred.  However,  when  a  product-oriented  test  is  pre¬ 
ferred,  then  the  cost/benefit  equations  need  to  be  modified. 

The  definition  for  "checkout"  used  thus  far  assumed  that  only  one  sol¬ 
dier  could  be  tested  at  a  time.  If  a  product  test  is  employed,  then  it  may 
be  possible,  where  resources  are  sufficient,  to  test  all  squad  members  simul¬ 
taneously,  just  as  when  a  written  knowledge  test  is  used.  Wherever  a  time 
cost  for  the  checkout  was  previously  involved,  the  cost  was  calculated  by 
multiplying  the  number  of  test  takers  by  C.  However,  for  a  simultaneously 
given  product  test,  the  cost  is  the  amount  of  time  that  represents  the  upper 
bound  of  the  slowest  acceptable  test  performance. 

As  before,  we  term  the  time  taken  by  the  entire  pretest  as  P.  However, 
in  formulating  the  credit  adjustment  for  passing  a  checkout  in  the  pretest 
that  would  have  cost  time  after  the  demonstration,  a  change  must  be  made. 

In  this  case  the  time  cost  of  the  checkout  at  either  time  is  simply  the  pro¬ 
ficient  group  size  (i.e.,  Nh)  multiplied  by  C.  To  avoid  possible  confusion, 
the  checkout  time  for  a  simultaneous  test  is  denoted  C ' . 

Pretest  Cost  *  -(NP  -  NhC'). 

The  corresponding  general  model  is: 
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B  =  NhD  -  NP  +  NhC '  -  Ne„,D,  or  N(D(h  -  e)  -  P  +  hC')  . 

N  N 

AN  EFFICIENT  DATA  COLLECTION  PROCEDURE 

The  alternative  pretesting  procedures  described  above  need  to  be  com¬ 
paratively  evaluated  from  the  standpoint  of  their  cost-effectiveness.  To  do 
this,  a  task  domain  must  first  be  specified,  as  for  example  the  56  common 
or  basic  tasks  included  in  the  Soldier's  Manual  for  Military  Occupational 
Specialty  (MOS)  11B10.  Given  the  fact  that  a  data  collection  effort  involv¬ 
ing  the  four  different  pretesting  procedures  when  applied  to  11B  tasks  would 
be  time-consuming  and  difficult  to  accomplish,  it  seemed  important  to  devise 
an  efficient  data  collection  procedure.  The  solution  adopted  was  as  follows. 
Participating  soldiers  are  asked  to  read  a  description  of  the  task,  condi¬ 
tions,  and  standards  for  each  task  that  is  sampled  from  the  Soldier's  Manual. 
Each  soldier  is  then  asked  first  to  estimate  his  ability  to  perform  it,  sec¬ 
ond  to  take  a  paper- and-pencil  test  regardless  of  his  estimate,  and  finally 
to  take  the  performance  test  regardless  of  his  written  test  result.  The 
full  procedure  is  explained  in  advance  so  that  soldiers  who  might  otherwise 
overestimate  their  capability  realize  that  a  hands-on  performance  test  will 
be  given.  The  data  collected  by  this  procedure  may  then  be  distributed  to 
all  four  active  pretesting  alternatives  by  means  of  a  logic  tree  analysis, 
thereby  effecting  a  sizeable  economy  in  data  collection  requirements.  The 
analytical  technique  is  illustrated  by  the  results  for  a  sample  task  in 
Figures  5  through  8. 


ILLUSTRATIVE  APPLICATION  OF  THE  COST-BENEFIT  MODEL 

Reported  here  is  a  des  .iption  of  an  application  of  the  general  cost- 
benefit  model  to  the  four  alternative  pretesting  procedures  that  have  been 
described  earlier  in  this  paper  in  Figures  1  through  4.  In  addition,  the 
model  is  applied  to  the  procedure  of  placing  all  soldiers  into  training 
without  pretesting.  This  application  is  based  on  data  that  were  collected 
from  infantry  soldiers  in  units  of  the  7th  Infantry  Division  at  Fort  Ord, 

Calif.,  during  1977.  The  data  were  specifically  collected  on  a  sample  of 
five  Soldier’s  Manual  tasks  included  in  the  set  for  Military  Occupational 
Specialty  (MOS)  11B.  These  five  tasks  were  selected  for  study  because  they 
provide  a  sample  which  covers  a  fairly  wide  spectrum  of  activities — leader¬ 
ship  skills  (Organize,  and  Employ  a  Tank  Hunter-Killer  Team),  cognitive 
(Encode/Decode,  and  Authenticate  Messages  with  a  KAL  16  Coding  Device) ,  and 
"hands-on"  (Emplace/Recover  an  M16A1  Anti-Personnel  Mine) .  This  small  sam¬ 
ple  of  tasks  clearly  cannot  be  used  to  estimate  accurately  possible  results 
for  all  tasks  in  the  11B  series,  but  it  does  serve  as  a  test  bed. 

The  data  were  collected  according  to  the  procedure  described  in  the  pre¬ 
vious  section.  Values  for  all  variables  except  A  (the  time  to  obtain  self¬ 
estimates)  are  shown  in  Table  1;  the  value  for  A  was  approximately  1  minute. 

The  logic  tree  analysis  was  used  to  obtain  the  values  for  model-specific 
variables  for  a  hypothetical  squad  having  nine  members  in  addition  to  the 
leader. 

By  employing  the  sample  data  shown  in  Table  1  to  the  alternative  pre¬ 
testing  procedures,  it  is  possible  to  generate  the  benefit  values  in  minutes 
for  each  of  the  sample  tasks,  as  shown  in  Table  2.  ; 

i 
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TABLE  1 

Empirically  Based  Data  for  Procedures  Permitting  False  "NO  GO"  Classifications 
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It  may  be  seen  from  the  summary  line  for  Table  2  that  Procedure  B  mini¬ 
mized  the  time  lost  (i.e.,  it  appears  to  be  the  most  cost-effective  procedure), 
while  automatically  placing  all  soldiers  into  training  appeared  to  be  the  most 
costly  procedure.  Procedure  B  happens  to  be  the  pretesting  procedure  selected 
by  the  new  training  system,  but,  of  course,  the  data  reported  here  are  insuf¬ 
ficient  to  establish  the  validity  of  this  selection.  With  a  sufficiently 
large  sample  of  tasks,  the  kind  of  raw  data  shown  in  Table  2  could  be  ana¬ 
lyzed  by  a  one-way  analysis  of  variance  test  to  estimate  if  the  alternative 
procedures  reliably  differ,  and,  if  so,  then  analyzed  by  a  test  like  the 
Newman-Keuls  to  estimate  exactly  which  procedures  are  distinctly  different 
in  their  benefit  values. 


A  COST-BENEFIT  ANALYSIS  FOR  SQUAD  LEADER  (TRAINER)  TIME 

Conducting  a  pretest  obviously  consumes  the  squad  leader's  own  time  as 
well  as  the  squad  member's  time.  In  this  section,  cost-benefit  models  for 
the  squad  leader's  time  in  his  role  of  trainer/tester  are  developed. 

The  time  that  a  pretest,  P,  takes  is  a  cost  for  the  trainer  which  needs 
to  be  adjusted  for  the  checkout  time,  C,  that  would  be  consumed  anyway  after 
the  demonstration/explanation  even  if  a  pretest  weren't  given.  The  adjust¬ 
ment  is  simply  the  number  of  hits,  Nh,  multiplied  by  the  time  for  each  check¬ 
out.  The  tentative  benefit  equation  for  the  squad  leader  is: 


B  -  -(P  -  NhC) . 

Li 

In  the  relatively  unusual  case  where  all  squad  members  are  hits,  then 
the  time  that  would  have  been  spent  on  the  demonstration,  D,  is  saved.  The 
appropriate  equation  is: 


B  =  -(P  -  NhC)  +  D. 

LD 

Given  the  infrequent  applicability  of  this  model,  it  will  not  be  dis¬ 
cussed  further  here. 

Since  the  squad  leader's  actions  affect  all  of  his  men,  it  may  be  ap¬ 
propriate  to  weight  his  time  by  the  number  of  his  men,  N.  The  benefit  equa¬ 
tion  then  becomes: 


B„L  =  -N(p  -  NhC). 

The  specific  formulation  of  B^l  for  each  of  the  alternative  pretest  pro¬ 
cedures  simply  requires  substituting  for  P  the  expressions  previously  ex¬ 
plained.  The  equations  for  each  procedure  are  as  follows: 

Procedure  A,  y  >  0:  B  =  -N(A  +  K  +  NwC  -  NhC) ,  or 

WL 

-N(A  +  K  +  NC (w  -  h) ) 
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Procedure  B:  B WL  =  -N  (A  +  NyC  -  NhC) ,  or 

-N (A  +  NC (y  -  h) ) 


Procedure  C:  B  =  -N(K  +  NwC  -  NhC) ,  or 
WL 

-N(K  +  NC(w  -  h)  ) 


Procedure  D:  B  =  -N(NC  -  NhC),  or 
WL 

-N2C(1  -  h) 


Procedure  E,  No  Pretest:  B^  =  0. 

To  illustrate  the  behavior  of  BWIi,  the  data  from  Table  1  have  been  used 
to  generate  results,  which  are  shown  in  Table  3.  It  can  be  seen  from  these 
results  that  the  least  costly  procedure  is  of  course  to  skip  the  pretest. 
Conversely,  the  data  show  that  the  most  expensive  procedure  would  be  to  give 
all  squad  members  a  checkout. 

The  experimental  training  system  places  a  strong  emphasis  on  the  use  of 
peer  trainers  to  motivate  acquisition  of  specific  task  and  leader  skills  by 
squad  members,  and  to  increase  the  squad  leader's  ability  to  spend  his  time 
where  he  can  be  most  productive.  Therefore,  it  is  possible  that  squad  lead¬ 
ers  may  generally  assign  one  of  their  squad's  members  who  has  passed  a  pre¬ 
test  to  conduct  the  task  demonstration.  In  this  case,  the  previously  defined 
model  for  Blq  applies,  since  the  time  that  the  demonstration  takes  is  saved 
by  the  squad  leader.  Table  4  displays  sample  weighted  squad  leader  benefit 
values  for  B^q  where  the  general  equation  is: 

BWLD  =  _N(P  ~  Nhc)  +  D- 

The  actual  application  of  this  equation  to  infantry  squads  for  tasks  with 
relatively  short  demonstration  times  may  well  be  limited  by  the  squad  leader's 
desire  to  provide  supervision.  However,  it  is  apparent  from  the  sample  BWLD 
values  shown  in  Table  4  that  rather  dramatic  time  savings  for  the  squad 
leader  may  be  generated  by  his  use  of  peer  instructors. 

In  the  next  section,  various  benefit  models  are  constructed  by  combining 
the  benefit  equations  for  squad  member  and  squad  leader  time. 


COMBINED  BENEFIT  MODELS  FOR  THE  SQUAD  LEADER  AND  HIS  MEN 
A  combined  general  equation  for  B  and  BWL  is: 

B  +  B  =  N(D(h  -  e  )  -  P  +  NCh2  -  P  +  NCh)  ,  or  N (D (h  -  ej  -  2P  +  NCh(h  +  D). 
WL  N  N 
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TABLE  3 


Since  the  overall  terms  which  reduce  P  {to  reflect  checkouts  that  would 
have  been  given  after  the  demonstration  even  if  the  pretest  were  not  used) 
constitute  only  a  fraction  of  P,  this  combined  model  provides  a  less  favor¬ 
able  outlook  for  pretesting. 

The  specific  equations  for  each  of  the  experimental  alternative  pretest¬ 
ing  procedures  are  presented  below: 

Procedure  A,  for  y  >  0: 

B  +  Bt_  =  N(D(h  -  e)  -  2  (A  +  K  +  Nwc)  +  NcH  (h  +  1)) 

WL  N 

Procedure  B: 

B  +  B  =  N(D<he„)  -  2 (A  +  NyC)  +  Nch(h  +  1)) 

WL  N 

Procedure  C: 

B  +  Br_  «  N(D(he  )  -  2{K  +  Nwc)  +  Nch (h  +  1)) 

WL  N 

Procedure  D: 

B  +  B„t  *  N(D(h  -  e  )  +  NC (-2  +  h2  +  h)) 

WL  N 

Procedure  E:  B  +  B  =0. 

WL 

The  corresponding  equations  for  (B  +  B^q)  are  found  simply  by  adding  ND 
to  the  (B  +  Bvjl)  values  for  Procedures  A,  B,  C,  and  D. 

For  certain  possible  applications  of  this  scheme  of  analysis,  it  may  be 
thought  inappropriate  to  be  giving  the  trainer's  time  a  weight  equal  to  the 
trainee  group  size.  For  example,  if  the  trainer  and  trainees  are  working 
full  time  at  training,  then  multiplying  the  trainer's  pretesting  time  cost 
by  the  trainee  group  size  may  lead  to  an  awkward  result.  Consider  that  when 
the  pretesting  benefit  for  the  squad  members  is  calculated,  the  time  that  the 
pretest  takes  is  multiplied  by  the  group  size,  N.  Now  when  Bjjl  is  calculated, 
the  pretest  time  is  also  multiplied  by  N.  In  this  way,  when  B  and  Bwl  are 
combined,  the  benefit  (or  cost)  of  the  pretest  time  for  the  group  is  effec¬ 
tively  counted  twice.  Therefore,  unweighted  equations  for  B l  and  BTn  may 
also  be  desirable  for  some  situations.  The  combined  general  equation  for 
Bj,  +  B  is: 

B  +  B  =  N (D(h  -  e  )  -  P  +  NCh2)  -  P  +  NCh,  or 
L  N 

N(D(h  -  e„)  -  P(N  +  1)/N  +  Ch(Nh  +  1)) 

N 

The  equation  for  B^p  +  B  is: 


2 

B  +  BT _  =  N(D(he„)  -  P  +  NCh  )  -  P  +  Nch  +  D,  or 
LD  N 

N(D(h  -  e  )  -  P(N  +  1)/N  +  ch(Nh  +1))  +  D. 

N 

To  conveniently  illustrate  the  relative  behavior  of  all  benefit  models, 
the  sample  data  from  Table  1  have  been  used  to  generate  results  (i.e.,  mean 
benefit  values,  and  rank  ordered  scores  for  the  means  from  each  model)  which 
are  shown  in  Table  5.  It  may  be  seen  that  Procedure  B  gave  the  best  overall 
performance  (x  =  4  man-minutes  saved)  and  Procedure  E  the  worst  (x  =  -76  man- 
minutes,  lost). 

If  the  sample  data  were  reliable,  then  the  following  conclusions  could 
be  made  for  the  infantry  tasks  that  were  studied.  When  only  the  squad  lead¬ 
er's  time  and  not  squad  member  time  is  considered,  and  when  the  squad  leader 
will  either  conduct  or  supervise  the  task  demonstration  (Models  Bl  and  B^) , 
then  no  pretesting  should  be  given.  But,  for  all  seven  remaining  models. 
Procedure  B  ranks  first  five  times  and  second  twice.  However,  these  conclu¬ 
sions  are  meant  only  to  illustrate  how  the  models  behave  and  may  be  analyzed, 
but  cannot  be  regarded  as  reliable  because  of  the  small  samples  of  tasks, 
soldiers,  and  testing  locations. 


CONCLUSIONS 

A  methodology  designed  to  enable  selection  of  an  optimally  efficient  pre¬ 
testing  procedure  for  use  in  group  training/testing  situations  has  been  ex¬ 
plained  and  illustrated  with  sample  results.  This  methodology  consists  of: 

a.  A  generalized  set  of  benefit  models, 

b.  Specified  alternative  pretesting  procedures, 

c.  Specific  algebraic  equations  which  permit  calculation  and  compari¬ 
son  of  the  benefits  accruing  from  the  various  alternative  procedures, 

d.  A  suggested  efficient  data  collection  plan  for  the  specified  alter¬ 
native  pretesting  procedures. 

One  salient  limitation  of  the  specified  alternative  pretesting  pro¬ 
cedures  and  their  benefit  models  derives  from  ah  ideal  requirement  imbedded 
in  the  proposed  on-the-job  infantry  training  system  underlying  this  research — 
namely,  the  requirement  that  task  training  may  officially  terminate  for  a  sol¬ 
dier  only  when  he  has  demonstrated  task  proficiency  by  passing  a  performance 
test.  As  a  consequence  of  this  requirement,  all  analyses  were  predicated  on 
the  assumption  that  no  soldier  would  falsely  be  classified  proficient  by  a 
pretest.  This  point  is  illustrated  by  the  diagram  below,  where  the  validity 
of  both  pretest  outcomes  is  described  as  a  function  of  the  soldier's  true 
proficiency: 
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TRAINEE'S  TASK  PROFICIENCY 


PRETEST  OUTCOME 


GO 

^  NO  GO 

GO 

VALID 

INVALID 

A 

B 

NO  GO 

INVALID 

VALID 

C 

D 

All  pretesting  procedures  were  constructed  so  that  cell  C  could  not  oc¬ 
cur;  the  only  mistakes  logically  possible  stem  from  cell  B.  However,  the 
ideal  requirement  that  training  always  be  capped  with  a  performance  test  is 
sometimes  perceived  as  a  time-consuming  luxury  when  training  is  conducted  by 
operational  units  in  field  settings.  Thus  in  reality,  the  invalid  outcome 
depicted  by  cell  C  may  sometimes  occur.  It  is  necessary  therefore  to  perform 
an  additional  analysis  which  will  enable  estimation  of  the  costs  or  benefits 
that  arise  when  performance  testing  is  not  reliably  practiced  in  a  training 
system  and  the  required  research  is  underway. 

A  final  point  to  consider  is  the  potential  scope  of  application  offered 
by  the  quantitative  approach  toward  cost-benefit  analysis  presented  here. 
Although  only  time  costs  have  been  discussed,  financial  costs  may  also  be 
handled. 

Time  may  be  directly  translated  into  money  by  multiplying  it  by  the  rate 
of  pay.  Once  time  is  translated  into  dollars,  then  other  financial  costs  may 
be  introduced  into  the  benefit  equations.  For  example,  if  expensive  simula¬ 
tion  equipment  needs  to  be  purchased,  then  its  ammortized  cost  may  be  in¬ 
cluded  in  the  cost-benefit  equation  which  represents  the  appropriate  alterna¬ 
tive  pretesting  (or  testing)  procedure.  Another  example  concerns  the  use  of 
expensive  ammunition  which  will  be  expended  during  performance  testing. 

The  potential  annual  dollar  equivalent  of  the  time  that  the  Army  may 
gain  by  the  use  of  pretests  can  be  illustrated  with  an  example  from  infantry 
training.  There  are  a  minimum  of  56  basic,  common  infantry  tasks  that  any 
soldier  in  a  rifle  or  weapons  squad  must  be  able  to  perform.  It  is  reason¬ 
able  to  assume  that  each  squad  would  be  required  to  train  on  each  of  the  56 
tasks  at  least  once  a  year.  The  mean  benefit  values  shown  in  Table  5  may  be 
multiplied  by  56  to  estimate  the  time  involved  for  a  single  squad  in  1  year, 
and  then  multiplied  by  the  number  of  squads  in  the  Army.  Finally,  the  time 
involved  for  squad  leaders  and  members  may  be  multiplied  by  their  estimated 
hourly  costs  to  find  the  dollar  equivalent.  As  an  example  of  the  annual 
benefit  that  may  be  achievable,  the  results  for  Procedure  B  may  be  compared 
with  Procedure  E  using  the  benefit  model  B  +  BLD.  In  this  comparison,  Pro¬ 
cedure  B  was  found  to  gain  about  430,000  man-hours  which  has  a  value  of  ap¬ 
proximately  6.5  million  dollars,  based  on  salary  costs  for  productive  train¬ 
ing  time  (estimated  at  30%  of  paid  time  by  Bialek,  1977) . 

More  important  than  the  annual  dollar  differences  between  the  various 
pretesting  procedures  is  the  percentage  of  time  that  may  be  freed  ot  lost, 
given  that  combat  readiness  depends  on  the  amount  of  training  time  that  can 
be  used.  A  comparison  of  Procedure  B  with  Procedure  E  for  Model  B  shows  the 
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difference  between  the  mean  benefit  values  to  have  been  (-42- (-136) ) ,  or 
94  minutes.  For  a  single  squad  member  this  amounted  to  94/9,  or  about 
10  minutes.  If  the  average  time  for  task  training  (i.e.,  task  demonstra¬ 
tion,  plus  skill  practice  until  mastery  is  achieved,  plus  the  checkout)  were 
1  hour,  then  the  time  saved  by  Procedure  B  over  Procedure  E  would  have  been 
about  17%.  Thus  if  the  magnitude  of  this  sample  result  is  accurate,  then 
training  efficiency  and  effectiveness  may  be  significantly  enhanced  by  ap¬ 
plication  of  this  quantitative  approach  toward  estimating  pretesting  costs 
and  benefits. 
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